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Abstract 
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We  define  a form  of  equivalence  between  Markov-renewal  processes  that 

includes  strong  and  weak  lumpability  as  special  cases,  and  examine  its  properties. 

If  {X  ,T  } is  a Markov-renewal  process  with  kernel  Q ( t ) and  {Z  ,S  } is  a 
n n n n 

Markov-renewal  process  with  kernel  Y(t),  then  it  is  shown  that  {X  ,T  } and 

n n 

{Z^.S^}  are  equivalent  if  and  only  if  there  is  a certain  homomorphism  between 
the  matrix  rings  generated  by  Q(t),  t e [ 0 , 00 ] and  Y(t),  t e [0,“] . The 
equivalence  is  identical  to  weak  lumpability  in  the  case  where  { Z^ , } is  a 
renewal  process. 

Although  the  conditions  for  strong  lumpability  can  be  written  in  an 
attractive  form,  they  are  too  restrictive  to  be  of  any  real  interest.  Weak 
lumpability  is  of  more  interest  since  (as  will  be  shown)  it  occurs  in  less 
trivial  examples,  but  the  necessary  conditions  are  very  complicated.  The 
equivalence  defined  herein  has  the  advantage  of  having  simple  necessary  and 
sufficient  conditions. 
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CHAPTER  1 


1.  Introduction.  A random  process  (xn>Tn)  n = 1,2,3,*  ••  with  Xr  taking  values 
in  a finite  or  countable  set  S (called  the  state  space),  and  taking  values 
in  [0,»]  is  called  a Markov  renewal  process  (MRP)  if 


P(Xn+l  = j’ 


•>v  ■ 


T J , <c  X ) 
n+1 — n' 


for  all  n e Z+,  j £ S,  t £ [0,°°].  Markov  renewal  processes  arise 

naturally  in  queueing  systems  and  since  renewal  processes  and  Markov  chains 

are  special  cases  of  MRP's,  a large  class  of  problems  in  the  study  of  random 

processes  can  be  handled  with  Markov  renewal  theory. 

Consider  the  departures  from  an  M/G/l/N  queue.  Let  Tr  be  the  time  between 

the  (n-l)St  and  n^  departure,  and  let  X be  the  number  of  customers  in  line 

n 

the  instant  after  the  nth  departure.  It  is  well  known  [3  ] that  (xn»Tn)  is  a 
MRP  on  a state  space  consisting  of  the  nonnegative  integers.  Now  consider  the 
special  case  where  G = M.  Since  the  M/M/1  /<*>  queue  is  an  M/G/l/N  queue,  the 
departure  process  is  a MRP  with  a countable  state  space.  By  [1]  and  [3], 
though,  we  know  that  in  steady  state  the  departure  process  from  an  M/M/1 


queue  is  a Poisson  process,  which  like  any  renewal  process,  is  a one  state  MRP. 
Thus,  in  some  sense,  the  infinite  state  MRP  that  represents  the  output  from  an 
M/M/1  queue  is  equivalent  to  a Poisson  process.  An  enormous  amount  of  work  has 
been  done  on  systems  with  M/M/1  queues  that  never  would  have  been  possible 
were  it  not  known  that  the  output  from  an  M/M/1  queue  is  a Poisson  process.  Any 
time  it  can  be  shown  that  a MRP  is  "equivalent"  to  a renewal  process,  the  amount 
of  computation  necessary  to  make  statements  about  the  process  will  be  drastically 
reduced.  This  paper  is  a first  step  towards  getting  such  results. 


■> 


When  two  random  processes  are  called  equivalent  in  this  paper,  it 
means  that  certain  specific  conditions  (to  be  given  later)  are  satisfied 
by  the  two  processes.  The  conditions  are  strong  enough  to  be  of  interest, 
and  weak  enough  to  assure  that  there  are  plenty  of  examples. 


2.  Lumpabilitv.  Probably  the  simplest  case  of  equivalence  between  MRP's  is 

lumpabilitv  in  Markov  chains  [8],  Let  {Xn}  be  a Markov  chain  on  a finite  or 

countable  state  space  S.  Let  A1,A2»  • ■ A be  a partition  of  S,  and  let 

F : S (A^,A2,  be  the  map  that  "lumps"  the  state  space  S onto  the 

partition  {A^,A9,  The  process  {F(Xn)}  may  or  may  not  be  a Markov 

chain.  In  general,  the  probability  of  going  from  A^  to  A^  in  {F(Xn>}  will 

depend  on  precisely  which  element  of  A.  the  {X  } process  is  in.  If  for  each 

i n 

i and  j,  though,  the  probability  of  going  from  A^  to  A^  is  independent  of  the 
State  in  A^  that  the  {Xn}  process  is  in,  then  the  process  (F(Xn)}  is  a Markov 
chain.  When  this  happens  we  say  {Xr}  is  strongly  lumpable  to  {F(Xn>}.  This  is 
a special  case  of  the  equivalence  to  be  defined. 

For  example,  say  S = {1,2,3}  and  let  { X^}  have  transition  probability 
matrix 


Let  F(l)  = A^,  F(2)  = F(3)  = A2*  The  process  { F ( X^ ) } is  a Markov  chain  on  (A^,A?) 
with  transition  probability  matrix 
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If  {X^}  is  strongly  lumpable  to  {F(Xn>}  then  no  matter  which  state  in  S 


the  process  starts  in,  {F(X  )}  will  be  a Markov  chain.  In  fact,  even  if  the 

n 

precise  state  that  the  process  begins  in  is  not  known,  the  ensuing  {F(X  )} 

n 

process  is  a Markov  chain. 

Sometimes,  even  though  {Xr}  is  not  strongly  lumpable  to  (FCX^)},  the  process 

{F(Xn)}  is  a Markov  chain  when  (Xn)  is  in  steady  state.  When  this  happens  we 

say  {Xn>  is  weakly  lumpable  to  {F(Xn>}. 

If  S is  a finite  set  with  m elements  and  F(S)  has  n elements  A, ,A„ . •••,A 

1*  2 n 

(n  < m) , then  the  following  mxn  matrix,  U,  can  be  constructed.  Let 


= 


( 0,  if  i 
( 1,  if  i 


Aj  * 

e A. 

1 


If  {X^}  has  a steady  state  then  there  is  a vector  II  that  satisfies  HP  = II 
where  P is  the  transition  probability  matrix  for  {Xn>.  Let  n be  an  n * m 


matrix  with 


n . . = 


0,if  j i k±  , 


n. 


I \ 


, if  j e A . 


k e A. 


i th 


The  i row  of  n is  the  conditional  probability  of  being  in  state  j given  that 
the  process  is  in  steady  state  and  that  the  process  is  in  A^. 
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Kemeny  and  Snell  [ 8 ],  show  that  {X  }is  strongly  lumpable  to  {F(X  )}  if 

n n 


and  onlv  if  PU  = U(npy),  and  that  if  {F(X  )}  is  a Markov  chain  then  its 
' n 


transition  probability  matrix  is  PPU.  They  also  show  that  IIP  = (npy)F  or 

PU  = y(nPU)  is  a sufficient  condition  for  {Xr}  to  be  weakly  lumpable  to  {F(Xn>}, 

For  example  let  S = {1,2,3}  and  set  F(l)  = F(2)  = F(3)  = A7.  Suppose  the 


transition  probability  matrix  for  {X  } is 

n 


P = 


A 


V 


3 

4 


In  this  case  n 


,111, 

v 3 * 3 9 3 


/ 


and 


U = 


V 


{X  } is  not  strongly  lumpable  to  {F(X  )}  since  P(F(X  ) =A, |F(X  ) = A~) 

T\  u n J.  Ti“’l  L 

depends  o:  whether  is  equal  to  2 or  3.  This  can  be  seen  formally  by  noting 

that  PU  # U(IIPU).  In  steady  state,  though,  {F(X  )}  is  a Markov  chain  since 


IIP  = (ilPU)n.  The  resulting  Markov  chain  {F(Xn>}  has  a transition  probability 


matrix 


FPU  = 


1/2 

1/2 

1/4 

3/4 

The  necessary  conditions  for  weak  lumpability  are  much  less  appealing  than 
the  necessary  and  sufficient  condition  for  strong  lumpability  or  the  sufficient 
conditions  for  weak  lumpability.  If  y is  a probability  vector  on  S then  define 


! 
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[y]  to  be  the  vector  of  conditional  probabilities  of  being  in  state  j 

(j = 1,2,  •••,m),  given  that  the  process  is  in  For  example,  the  i1"*1  row 

of  the  matrix  II  is  [”|\  Let  be  the  set  of  all  finite  sequences  of  states 

in  F(S)  that  end  w’ith  A..  If  A.  , A.  , •••  A.  , A.  and  A.  , A.  , •••  A.  , A. 

J V *2  V 2 31  32  3q  3 

are  two  elements  of  F then  for  {Xr}  to  be  weakly  lumpable  to  {FCX^)}  it  nus: 
be  true  that  for  each  a e(l,2, 


l l P.ff  y.1  - I y P y. 

i6  i , , i£  1 


i e S B e A 


i e S « £ A 


where 


and 


1 ^1  X2  13  ^'k  3 

y = [ •••  [ t [ [ [n]  p]  ‘ p]  J p]  4 •••  p]  k p] 


5 3 j 

y = t •••  [mm  - p]  - p]  J p]  41  •••  P]  q p] 


Serfozo  [10]  showed  that  strong  and  weak  lumpability  can  be  defined  for 

MRP's  in  an  analogous  manner.  In  fact,  the  conditions  for  strong  and  weak 

lumpability  in  MRP's  are  virtually  identical  to  the  conditions  for  Markov  chains. 

If  {X  ,T  } is  a MRP  on  a finite  state  space  S = {1,2,3,  •••m},  with 
n n r 

kernel  Q(t)  (i.e.  (^(t)  = p(xn+i  * 3.  Tn+1  <_  t|XR  = i))  and  F : S - {A1,A2,---  ,An 

is  a partition  of  the  state  space  then  {Xn,Tn}  is  said  to  be  strongly  lumpable 
to  (F(Xn),Tn)if  {F(Xn),Tn}is  a MRP. 

Again,  let  n be  the  steady  state  vector  for  the  embedded  Markov  chain  • 

(i.e.  nQ(®)  “ F.) , and  let  H,U  be  defined  as  before.  Serfozo  shows  that 
{Xn,Tn>  is  strongly  lumpable  to  {F(Xn),Tn}  if  and  only  if  Q(t)U  ® U(nQ(t)U) 
for  all  t t [0,®].  Likewise  if  for  all  t,  Q(t)U  ■ U(FQ(t)y)  or  IIQ(t)  = (IIQ(t)  U) II 
then  {F(Xn),Tn}  is  a MRP  in  steady  state  (i.e.  weakly  lumpable).  Unfortunately, 
the  necessary  conditions  for  weak  lumpability  are  again  very  complicated. 
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Let  Tj  be  the  set  of  all  finite  sequences  of  states  in  F(S)  that  end  with  . 

If  A.  , A.  , •••A.  , A.  and  A.  , A.  , A.  . A.  are  two  elements  of  T.  and 

11  x2  V 3 31  32  3q  3 3 

{t^,  t9,  • • • , t , {s^,  s9,  •••,s  } are  two  sequences  of  positive  real  numbers 

then  for  {Xn>Tn}  to  be  weakly  lumpable  to  {F(Xn),T^}  it  must  be  true  that  for 

each  a z (1,2,  •••,n)  and  t e [ 0 , 00 ] , 


..  I o „ L Qi6(t)  y±  m A*  A * Qie(t)  Yi 


i e S 8 e A 


i e S 8 e A 


where 


Y1  = [ • • • [[[[["]  1 QC^)  ] 2 Q(t2)  ] 3 Q(t3)  ] 4 * • • Q(tk_1)  ] k Q(tk)  ]' 


y2  = [ •••  [ [ [ [ [ n] 3 1 Q(s1)]J2  q(s2)]J3  q(s3) ]34  •••  q(s  )]3q  Q(sq) ]J . 


In  this  paper  a type  of  equivalence  will  be  defined  that  includes  all  of 
the  cases  discussed  so  far  and  has  the  added  property  that  a necessary  and 
sufficient  condition  for  two  MRP's  to  be  equivalent  can  be  written  in  a simple 
form. 


3.  Definitions  and  Preliminaries.  The  following  concepts  will  be  used 
throughout  this  paper. 

Definition  1.1.  Let  V be  a vector  space,  and  let  T : V •*  V be  a function  on 
V.  A subspace  W of  V is  said  to  be  invariant  with  respect  to  T if  for  all 
w £ W,  Tw  z W. 

For  example,  say  V is  Rn  and  T(v)  = Av  where  A is  an  n * n matrix.  If 
wl’w2’  ’ ’ *»  wk  are  eigenvect°rs  A,  then  the  space  It1  spanned  by  w^,w2>  ’"",wk 
is  invariant  under  T. 
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Definition  1.2. 
operations  + , • that 


A ring,  R is  a collection  of  objects  along  with  two 
satisfy  the  following  properties:  y a,b,c  t R, 


(1) 

a + b e 

R 

(2) 

a + b = 

b + 

a 

(3) 

(a  + b) 

+ c 

= a + (b  + 

c) 

(4) 

3 0 e R 

that 

satisfies 

a 

+ 0 = a 

(5) 

3 - a that  satisfies  a 

+ 

(-a)  = 

(6) 

a • b e 

R 

(7) 

a • (b-< 

c)  * 

(a-b)  • c 

(8) 

a • (b  ■ 

*■  c) 

= (a-b)  + 

(a- 

c) 

(9) 

(b  + c) 

• a 

= (b-a)  + 

(c* 

a) 

The  operation  • need  not  be  commutative  in  a ring.  For  example,  the  set 
of  all  n x n matrices  is  a ring.  In  this  paper  we  will  be  interested  in  various 
subsets  of  the  set  of  all  n x n matrices  that  retain  all  the  ring  properties. 

Definition  1.3.  Let  { a i t I,  be  a collection  of  elements  of  a ring,  R. 
The  ring.  A,  generated  by  {ai } is  the  smallest  subring  of  R that  contains  all 
of  the  {a^}. 

For  example,  consider  the  ring  of  integers  Z.  The  ring  generated  by  {2  } 
is  the  ring  of  even  integers.  For  a less  trivial  example  consider  the  ring, 

N,  of  n x n matrices.  Let  M^,M2,  • • • , be  elements  of  N.  A typical  element, 
of  the  ring  generated  by  • • • , might  be 


Definition  1.4.  Let  R^  and  R^  be  two  rings  and  let  T : - R,  be  a map. 

T is  called  a ring  homomorphism  (homomorphism)  if,ya,b  e R^, 
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(l-)  T(a-b)  * T(a)  ■ T(b)  and 
(2)  T (a  + b)  = T(a)  + T(b) . 

Consider  the  following  example  of  a ring  homomorphism.  Let  y be  some 
vector  in  Rn  and  let  R^  be  the  set  of  all  n x n matrices  that  have  y as  a 
left  eigenvector.  If  A £ R^,  define  T(A)  » o where  a is  the  eigenvalue  of  A 
associated  with  y.  Thus  T(A  + B)  *=  T(A)  + T(B)  and  T(AB)  c T(A)T(B)  so  T is 
a homomorphism  from  R^  to  R. 

In  this  paper  we  will  only  consider  a special  class  of  MRP's  defined  as 
follows. 

Definition  1.5.  An  m state  MRP,  {X  ,T  }.  1 < m < <*.  with  kernel  Q(t) 

n n - ~ 

will  be  called  simple  if 

(1)  (t)  is  nonnegative,  nondecreasing  and  right  continuous, Vi, j , 
m 

(2)  l Q,,(»)  = 1,  Vi, 

j-1  J 

(3)  Q±j  Ct>  =0,  Vt  e (-“,0),  V i, j , 

(4)  3!'  H e R™  that  satisfies 
(4a)  II  Q(»)  « n 

(4b)  HU  = 1 

(4c)  lim  Qn(<=)  = UH 
n -►  00 

where  U “ (1,1,  ••*,1)^'. 
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Conditions  (1),  (2),  and  (3)  assure  that  Q(t)U  is  a column  of  nonnegative 
distribution  functions.  Condition  (4)  is  equivalent  to  requiring  that  the 
embedded  Markov  chain  {X^}  is  irreducible,  aperiodic  and  recurrent  non-null. 
(See  [2]  for  proof  of  this  assertion  and  other  similar  results.) 

Let  { } be  ar.  n state  MRP  with  kernel  Q(t)  and  say  the  initial 
distribution  on  the  state  space  is  II.  The  following  quantities  are  of  interest 


(1)  < t|XQ 


(2)  P^  < t) 


i) 


(3)  P(TX  £ tr  T2  £ t2, 
We  can  write  (1)  as 


•,  T < t ) 
m — m 


(4)  P(TX  < t|XQ  = i) 


l = j,  T < t |X0  = i)  * (Q(t)U) 

j-1  1 


Thus  the  column  vector  Q(t)U  is  a vector  of  probabilities  of  a transition 
before  time  t given  the  initial  state.  To  solve  for  (2),  we  weight  each 
initial  state  by  the  initial  probability  distribution,  so 

(5)  P(TX  < t)  = HQ(t)U. 

The  i—  element  of  the  vector  IIQ(t)  is  the  probability  that  starting 
with  the  initial  distribution  H,  the  process  has  its  first  transition  before 
time  t,  and  the  transition  is  to  state  i.  Likewise  the  i—  element  of  the 
vector  IIQ(t^)Q(t0)  is  the  probability  that  starting  with  the  initial 
distribution  L,  the  process  has  its  first  transition  before  time  t^,  its 
second  transition  in  less  than  t2  time  units  after  the  first  transition,  and 
ends  up  in  state  i.  Inductively,  we  obtain 

(6)  P(TX  £ tr  T2  £ t2,-  • • ,Tn  £ tm)  - XQ(t1)Q(t2)  • • • QU^U. 
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In  Chapter  II  we  define  equivalence  between  MRP’s  and  investigate  its 
properties.  Sections  1-4  deal  with  the  important  special  case  where  the 
equivalence  is  between  a MRP  and  a renewal  process  (a  one  state  MRP) . 
Section  5 deals  with  equivalence  between  finite  state  MRP's. 


CHAPTER  II 


* 


1.  Equivalence.  A recurrent  renewal  process  { } is  a sequence  of  independent 
and  identically  distributed  nonnegative  random  variables  with  sn  < “ with 
probability  one.  The  sequence  {S^}  can  be  thought  of  as  the  times  between  some 
fixed  event  that  occurs  repeatedly.  Associated  with  each  MRP,  {X  ,T  ),  is  a 
sequence  {T  }.  Suppose  {Xn>Tn}  is  a simple  MRP  on  a finite  state  space 
S = {1, 2, • ’ • ,N}  with  kernel  Q(t) . If  Y is  the  initial  distribution  on  S then 

(1)  P^  <_  t)  *=  yQ(t)U  where  U = (1,1,  ‘".l)1,  and 

(2)  P(TX  £ tx,  T2  £ t2,  •••,Tn  <_  tn)  *=  YQ(t1)  Q(t2)  •••  Q(tn)U. 

Let  H be  the  steady  state  vector  associated  with  the  embedded  Markov  chain 

(X  },  and  define  r(t)  ■ HQ(t)U. 
n 

Lemma  2.1.  r(t)  is  the  cumulative  distribution  of  some  nonnegative 
random  variable. 

Proof . From  conditions  (1)  and  (3)  of  Definition  1.5,  r(t)e0  if  t < 0 
and  r(t)  is  nondecreasing  and  right  continuous.  From  condition  (2)  we  have 
r(“)  = nQ(°°)U  * "U  * 1 so  r(t)  is  a distribution  function  of  a nonnegative 


random  variable.  q 
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For  the  remainder  of  this  chapter  all  MRP's  will  be  assumed  to  be  simple, 
and  all  the  renewal  processes  will  be  recurrent.  The  following  theorem 
motivates  the  definition  of  equivalence  between  a MRP  and  a renewal  process. 

Theorem  2.2.  If  {X  ,T  } is  in  steady  state  then  {T  } is  a renewal  process 
n n n 

if  and  only  if  Vm,  Vtpt,,  HQ(t^)Q(t2)  ••Q(tjn)U  ■ r(t^)r(t2)  ••r(tBi). 

Proof.  ( -~  ~^ ) if  {T^}  is  a renewal  process  then  P(T^  <_  tp  T2  £ t2’"  Tm  — Cm^ 
- Pd^  £ t1)  PCTj^  <_  t2)*'*  Pdj^  £ tffl) . But  this  says 

i:Q(T1)  Q(t2)  • • *Q(tm)U  = r(t1)  r(t,,)  • • -rC^) . 


( <i--' ) We  must  show  that  {Tn)  is  a sequence  of  nonnegative  independent  and 
identically  distributed  random  variables.  Since  Q(t)  £ 0 it  is  clear  that  {Tn; 
is  nonnegative.  Also  r(t)  is  not  a function  of  n,  so  it  suffices  to  show  that 
{T^}  is  a sequence  of  independent  random  variables.  Let  ipip  "*in  be  ary 
n positive  integers.  We  must  show  that 

P(T  <t.  , T,  <t  .---.T.  < t,  ) = P(T  <t,  ) P(T±  <t.  ) P(T.  <t.  ) . 
h 11  i2  12  xn  in  il  il  i2  *2  ^ *n 


Since 


'ill'll-  1 V^n  i V 


where 


t.  , if  j = i, 

\ K 


• if  j i (ipi2»  ‘ ’ in)  » 


we  have  ^l-h*  T^tp*-’,^  £ t±^)  ■ r(tx)  r(t2)  •••  r(tin>.  But 

rCtj^)  r(t2)  •••  r(tin)  = r(ti;L)  r(ti2)  ••  r(tin)  since  r(«>)  = 1.  Thus  {TnJ  is 
a renewal  process.  □ 


3 


From  no w on  the  following  notation  will  be  used.  The  symbol  0 will  denote 
a simple  MRP  with  kernel  Q(t).  The  set  of  matrices  {Q(t)},  t e [0,"°],  along  with 
an  initial  probability  distribution  on  the  state  space,  describe  the  MRP  in 
question,  since  from  them  it  is  possible  to  determine  all  transition  probabilities. 
Thus,  there  is  no  ambiguity  in  using  the  symbol  Q to  denote  a MRP.  Likevdse,  the 
symbol  r denotes  the  renewal  process  with  distribution  r(t)  without  ambiguity. 


*? 


max  QnT1(«)  - T. . ' < c/m.  Thus  the  jtn  column  of  Qn  (“)  is  (a,  ,a_, . . . ,a  ) 
n i ijm 

i.  j 3 3 


where  a.  - 2,  ' < z/m  for  each  i.  Thus 
i J 

!(vQn_1(“)  -s),!-  i(?Vk>  - n3 


V 


£ I \!ak  - nJ  1 £/“  I Yk  ' z/m- 
k-1  J k«l 

This  says  that  each  element  of  the  vector  yQn  ^(“O  differs  from  the 
corresponding  element  of  n by  less  than  e/m.  By  hypothysis, 

YQ(tl)Q(t2)--Q(tn)U  = f(t1)f(t2)»*f(tn),  so  by  letting  ^ « t2  - • • - * - 

we  get  YQn-1(®)  Q(tj)U  - f (tn) . Let  YQn_1(°°)  - Fi'.  To  show  that  f(t)  - r(t)  it 
suffices  to  show  that  |H'Q(tn)U  - IIQftn)U|  is  small. 

|iI'Q(t  )U  - i”Q(tn)U|  - in  - r|Q(tr)U  <_  e/m  UTQ(tn)U  £ e/m  UTU  = e. 

Since  e was  arbitrary  we  have  r(t)  - f(t). 

00  00 

(case  2:  m-“).  Fix  e and  choose  N such  that  £ y.  < e/7,  I H.  < e/7. 

i-N+1  1 i-N+1 

Since  Q is  simple  we  have  lim  Q^C00)  - H Vi,  so  we  can  choose  M such  that  n > M 
implies  | Qij  C00)  — | < e/7N  for  i,j  £ N. 

We  must  show  |f(t)  - r(t)|  is  small,  Vt.  Since  f(t)  - YQn(0D)Q(t)U  for 
any  value  of  n,  it  suffices  to  show  that  for  n > M,  | YQn(“)Q(t)U  - r(t) | is 
small,  Vt.  Since  r(t)  - IlQ(t)U, 

00 

I YQn(“')Q(t)U  - r(t)  | < |YQn(°°)  - n|Q(t)U  <_  l I YQn(“)  - 

J-l  3 

N 

- I ! YQn(°°)  - EL  + I |yq“(-)  - HI. 

j-l  3 j-N+1  J 

We  have, 

N N » « N 

l lvQn(®)  - n|.  < l l y±|q“  (-)  - nJ  - [ y.  I |q"  (-)  - n | 

j-l  3 j-l  i-l  3 3 i-l  1 j-l  3 J 
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Theorem  2.4.  Let  Q be  a MRP  with  steady  state  vector  II,  and  let  r be  a 
renewal  process.  If  Vt,  HQ(t)  - r ( t) II  then  Q-r. 

Proof.  If  HQ(t)  «=  r(t)H  then  Vn,  t^,  1 2,  • • •,  t , 

RQ(t1)  Q(t2)---  Q(tn)U  - r(tx)  HQ(t2)  •••  Q(tn)U 

■rC^)  r(t2)  •••  r(tn)UU  «■  r^)  r(t2)  •••  r(tn).  q 
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r w 

Theorem  2.5.  If  Vt,  Q(t)U  * Ur(t)  then  0-r. 

Proof.  If  Q ( t ) U = U r ( t ) then  Vn  , t^,  t„  t , 

HQ(t1)  Q(t2)  •••  Q(tn)U  = HQ(tl)  Q(t2)  •••  Q(tn_1>U  r (tR) 


= nur(t.)  r (t_)  • • • r (t  ) » r(t,)  r(t,)  •••  r(t  ).  Q 
1 z n l £ n 

Theorems  2.4  and  2.5  are  special  cases  of  the  sufficient  conditions  for 
weak  lumpability  that  Serfozo  gives  in  [10].  Theorem  2.4  says  that  if 
the  steady  state  vector,  H,  is  a left  eigenvector  of  Q(t)  for  every  t then  Q-r, 
where  r(t)  is  the  eigenvalue  of  Q(t)  corresponding  to  the  eigenvector  H.  Notice 
that  in  theorem  2.5  it  was  not  important  that  the  starting  vector  was  II.  Any 
vector  that  satisfied  YU  * 1 would  have  worked.  This  is  because  Q(t)U  = Ur(t) 
is  a necessary  and  sufficient  condition  for  strong  lumpability  of  Q to  r. 

Theorem  2.5  says  that  if  the  row  sums  of  the  matrix  Q(t)  are  the  same  for  all 
t then  Q-r  where  r(t)  is  the  common  value  of  the  row  sums  of  Q(t).  If  the  row 
sums  of  Q(t)  are  the  same,  then  no  matter  which  state  the  process  is  in,  the 
time  until  the  next  transition  has  the  same  distribution.  Thus,  knowing  the 
state  that  the  process  is  in  gives  no  extra  information  about  the  time  until 
the  next  transition.  It  is  clear  that  in  such  cases  the  times  between  state 
transitions  is  a renewal  process.  The  intuitive  justification  for  theorem  2.4 
is  less  obvious,  but  most  interesting  cases  of  equivalence  seem  to  be  of  that 
type.  We  will  see  later  that  Burke's  theorem  is  a simple  corollary  of  theorem  2.4. 


Let  Q be  a MRP  with  n states  (1  < n <_  ®)  , and  steady  state  vector  II.  We 
define  the  following  subsets  of  Rn. 

I 
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Definitions  2.2. 

(A)  Let  S = {v  e Rn:  vQ(t)U  - (vU)  r(t),  Vt}  where  r(t)  - HQ(t)U. 

(B)  Let  V be  the  largest  subset  of  S that  is  invariant  under  multiplication 

by  Q(t)  (i.e.  v e vQ(t)  c V ) • 

(C)  Let  P - {v  e Rn:  v > 0,  vU  * 1}.  P is  the  set  of  probability  vectors. 

(D)  Let  K = UnP  . 

Lpirma  2.6.  S and  1/  are  subspaces  of  Rn.  K is  a compact  and  convex  set 
if  n < “. 

Proof . Clearly  {11}  e S and  {0}  e S,  so  S has  at  least  two  elements.  Say 

Yp  y2  e Then  YjQ(t)U  “ r(t)(YjU)  and  Y2Q(t)U  “ r(t)(y2U).  Thus 

(a  Y1  + bY,)Q(t)U  = r(t)  (a  Yj +b  y2)U  so  a Yj_  + b y2  £ S.  Let  W be  any  invariant 

subset  of  S and  let  W*  be  the  subspace  generated  by  W.  Say  w e fc1*. 

k k k 

w * j a.  v.  where  w.  e W 1*1,2,  * k.  But  vQ(t)  * J a.  w.Q(t)  * I a.  v! 

i-1  1 1 1 i-1  i-1 

where  w^  e W since  W is  invariant.  Thus  wQ(t)  c W*,  so  W*  is  invariant.  Since 

1/  is  defined  to  be  the  largest  invariant  set,  it  must  be  a subspace. 

The  set  P of  probability  vectors  in  Rn  is  closed  convex  and  bounded.  So  is 

PHI/  since  l/is  a subspace,  thus  K is  compact  and  convex.  O 

Consider  the  column  vector  Q(t)U.  If  v c S then  ~(^j)  Q (t)U  “ JIQ(t)U,  Vt. 

This  says  that  ( - II)  Q(t)U  * 0,  Vt,  which  says  that  the  vector  - I* 

is  orthogonal  to  the  vector  Q(t)U,  Vt.  Thus  we  have  the  following  important 
lemma . 

Lemma  2.7.  If  Q is  an  n state  MRP  and  there  exists  times  t^,t2,  •••  t^ 
such  that  {Q(t1)C,  Q(t2)U,  Q(tn)U>  is  a linearly  independent  set  then  S 
is  the  subspace  of  Rn  generated  by  {11}  . 
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Proof.  If  v £ S then  ( _ -)  nust  be  orthogonal  to  each  Q(t)U.  But 

only  the  zero  vector  can  be  orthogonal  to  n independent  vectors  in  R . Thus 
the  only  elements  of  S are  the  points  on  the  line  through  T.  and  the  origin.  O 

We  now  show  the  importance  of  the  sets  l7  and  K. 

Theorem  2.8.,  Q-r  <f—  *■»  H e l/. 

Proof.  (=5>-)  If  Q-r  then  Vn  t.  , t, , * • • , t , TlQ(  t.  ) Q(t_)  0(t  )U 

, 1 z n i z n 

* rCtj^)  r(t2)  •••  r(tn)*  Say  nQ(t1>  Q(t2>  •**  Q(tR)  i ■$.  Then  3 1 such  that 

HQCt, ) Q(t,)  • • • Q(t  ) Q(t)U  i (HQ  (tn ) •••  Q(t  )U)r(t).  But  this  savs  that 
l z n in 

Q+r.  Thus  Vn,  t.,t,,**',t  we  have  HQ(t. ) Q(t_)  •••  Q(t  ) e S.  Let 
i z n i z n 

It'  «=  {w:  w - II  Q(t^)  Q(t?)  •••  Q(tn)  for  some  n.t^tj,  }.  W must  be  a 

subset  of  S,  and  clearly  ll!  is  invariant  under  multiplication  by  Q(t).  Thus 

i'J  O V . But  E c W since  HQ(°°)  ■ U,  so  U z l/. 

( ^ ) If  II  t V then  U c K.  Let  v e K.  Since  each  element  of  Q(t)  is 

nonnegative,  vQ(t)  > 0.  Also  vQ(t)  “ v'  where  v'  e V.  Finally,  vQ(t)U  *r(t) 

since  v e K so  v'U  = r(t).  This  implies  that  v(^-^y  ) e K,  so  K is  invariant 

under  multiplication  by  Q(cVr(t).  Thus  HQ(t. ) Q(t.)  •••  Q(t  )U  ■ r(t  ) r (t0)  • • • r (t  ) . 

JL  z n i.  z n q 

Corollary  2.8.1.  Q-r  ^=>E  c K. 

Proof.  If  H £ K then  E t 1/  so  0-r.  If  Q-r  then  II  e V . But  ITU  = 1 so 
n s K.  a 

Theorems  2.4  and  2.5  were  sufficient  conditions  for  Q-r  which  are 
relatively  easy  to  use  in  practice.  Theorem  2.8  gives  a necessary  and 
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sufficient  condition  but  there  is  not  yet  any  simple  way  of  determining  what 
l'  is.  The  following  theorem  is  a necessary  condition  for  Q-r  which  is  also 
useful  in  practice. 

Theorem  2.9.  If  Q is  a finite  state  MRP  then  Q-r  =^r(t)  is  an 
eigenvalue  of  Q(t),  Vt. 

Proof . If  Q-r  then  II  c K,  so  K is  not  empty.  We  have  shown  that  if  Q 

is  finite  dimensional  then  K is  compact  and  convex  and  invariant  under 

multiplication  by  Q(t)/r(t).  Thus  by  the  Brower  Fixed  Point  Theorem  [9], 

Vt,  3 >'t  such  that  y£  £ K and  Yt(Q(t)/r(t))  « This  says  YtQ(t)  = r(t)Yt 

so  r(t)  is  an  eigenvalue  of  Q(t).  o 

Corollary  2.9.1.  Q-r  =*>•  det(Q(t)  - r (t)I)  = 0,  Vt. 

Proof.  This  is  just  a restatement  of  the  theorem.  a 

Although  it  is  unrealistic  to  check  to  see  whether  r(t)  is  an  eigenvalue 

of  each  Q(t),  theorem  2.9  says  that  one  can  show  that  Q+r  by  merely  finding  a 

value  of  t where  r(t)  is  not  an  eigenvalue.  The  following  theorem  is  useful 
in  the  same  way. 

Theorem  2.10.  If  Q is  an  n state  MRP  (n  < «>)  and  3ti»t2,  " t such 
that  {Q(t^)U,  Q(t2)U,  ••*,Q(tn)U}  is  a linearly  independent  set  then 

Q-r  <*=>nQ(t)  «=  r(t)n,  Vt. 

Proof.  By  lemma  2.7,  5 is  a one  dimensional  subspace,  so  either  1/  = 5 
or  1/  = {0}.  By  theorem  2.8,  Q-r  ss  .ft  n e 1/  so  K must  consist  of  the  single 


vector  {II}.  Since  K is  invariant  under  multiplication  by  Q(t)/  r(t)  we  have 
IlQ(t)  ■ r(t)n.  The  converse  is  a restatement  of  theorem  2.4.  □ 
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Using  the  results  so  far,  a rough  algorithm  for  determining  whether  a 

MRP  is  equivalent  (or  weakly  lumpable  ) to  a renewal  process  can  be  formulated. 

First  of  all,  if  a MRP,  Q,  is  equivalent  to  a renewal  process,  the  renewal 

process  it  is  equivalent  to  must  be  r where  r(t)  = PQ(t)U,  and  H is  the  steady 

state  vector  of  the  embedded  Markov  chain  Q(°°).  The  first  step  should  be  to 

try  to  show  that  Q*r  since  in  general  that  will  be  the  case.  If  Q is  an  n 

state  MRP  choose  t^,t2,  •••,tn  randomly  or  otherwise  and  compute 

{Q(t.)U,  • • • ,Q(t  )U).  If  {Q(t.)U,  •••,Q(t  )U)  is  a linearly  independent  set 
in  in 

(as  it  will  be  in  general) , then  Q-r  if  and  only  if  K is  a left  eigenvector 
of  each  Q(t).  Choose  some  t and  perform  the  multiplication  IiQ(t).  If  this 
product  is  not  r(t)II  then  Q/r.  If  IlQ(t)  = r(t)H  the  chances  are  it  was  not  a 
coincidence.  See  if  I!  is  indeed  a left  eigenvector  of  each  Q(t)  by  writing 
out  explicitly  RQ(t)  as  a function  of  t.  If  H is  a left  eigenvector  of  each 
Q(t)  then  Q-r. 

If  {Q(t,)U,  •••>Q(t  )U}  was  not  a linearly  independent  set,  it  becomes  more 
1 n 


complicated.  If  dim{Q(t^)U,  ’"■*Q(tn)U}  = 1 check  tc  see  if  U is  a right 
eigenvector  of  each  Q(t)  by  summing  the  elements  of  the  rows  of  Q(t).  If  the 
row  sums  of  Q(t)  are  the  same  for  all  t , then  Q is  strongly  lumpable  to  r which 
implies  equivalence. 

Another  way  of  trying  to  quickly  show  that  Q/r  is  as  follows.  Choose  t^ 
and  t2  and  perform  the  multiplication  (n  - )Q(t2)U.  If  II  e K then, 

so  is  IlQ(t2)/r  (tj) . But  for  any  v^  and  v2  in  K,  (vi-v2)Q(t)U  - 0 Vt. 

Thus  if  the  result  of  the  multiplication  is  not  zero  then  Q*r. 

If  no  conclusion  has  been  reached  yet  then  either  try  different  values 
of  time  or  attempt  to  find  the  subspaces  S and  U.  Finding  S is  easy,  finding 
V is  much  more  difficult.  The  next  section  gives  a more  abstract  description  of 


1/  and  K. 
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3.  Equivalence  as  a Homomorphism.  Let  Q be  an  n state  MRP  (1  < n _<  “>)  with 
steady  state  vector  H.  Associated  with  each  t e [0,»]  is  a matrix  0(t)  . 

Let  0 be  the  ring  of  matrices  generated  by  { Q ( t ) },  t e [0,°°].  For  each 
crobability  vector  y in  Rn  we  have  a map  F : 0-*-  R where 

' i 

■ 

(4)  Fy(A)  =YAU. 

A necessary  and  sufficient  condition  for  equivalence  can  now  be  written  in  a 
very  simple  form. 

Theorem  2.11.  Q-r  )>  F^.  is  a homomorphism. 

Proof.  ( =>)  If  Q-r  then  Vn,  t^.t.,,  • • •,  HQ(t^)  Q ( 1 2 ) * • • Q(tn)U  = 

(nQ(t1)U)(nQ(t,)U)  •••  (IIQ(tn)U).  Also  if  A1,A2  e 0 then  Il^+A^U  = JIA^IM-T^U 
so  F^  is  a homomorphism. 

(<*  ) If  F_  is  a homomorphism  then  Vn,  t,,t9,  , 

n i n 

nq(t1)  Q(t2)---  Q(tn)U  - (HQ(t1)U)(nQ(t2)U)---(nQ(tn)U)  = r(t1)r(t2)---r(tn), 
so  Q-r.  O 


Theorem  2.11  says  that  the  question  of  whether  or  not  a MRP  is  equivalent 
to  a renewal  process  is  identical  to  the  question  of  whether  a certain  map 
from  a matrix  ring  to  the  real  line  is  a homomorphism.  If  there  were  a good  way 
of  deciding  when  there  are  homomorphisms  between  matrix  rings  and  the  real's 
the  analysis  of  equivalence  between  simple  MRP's  and  renewal  processes  would  be 
complete.  This  is  certainly  a deep  question  that  will  demand  more  research. 

The  maps  {F  } also  give  an  alternate  characterization  of  the  sets  K and  l/. 


Theorem  2.12. 


y c K 


F is  a homomorphism. 


P 


v> 


Proof . ( ~ *>.1  If  y e K then  F t K , so  Q-r.  Thus  F„  is  a homomorphism. 

But  KCS  , and  from  the  definition  of  S,  Fj,  * F^. 

(<*=)  Let  T “ {v;vQ(t)U  = (vU)  YQ(t)U},  and  let 

W*  {w:w  = yO(t1)  Q(t2)  • • • Q(tR)  for  some  n.t^t^  • • •,  tn>. 

tl?  is  a subset  of  T;  for  if  not,  there  would  be  some  t^.tj,  such 

that  yQ(t. ) Q(t_)  * * • Q(t  ) t T.  But  this  would  imply  that  for  some  t, 

1 l n 

yQ(t])Q(t2)  • • • Q(tn)  Q(t)U  i (yQCtj^)  • • • Q(tn)U)  (yQ(t)U)  which  says  that  F^  is 

not  a homomorphism.  The  steady  state  vector  must  be  in  the  closure  of  U' 

since  II  = lim-  yQn(°°),  thus  Ve  > 0,  3w  e It1  that  satisfies  |w-n|U  < e. 
n -*•’» 

Also  w can  be  chosen  so  that  wU  = 1 since  yQn(®)U  = l,  Vn.  Thus 

| HQ(t)U  - YQ(t)u|  - I nQ(t)0  - wQ(t)U  | = | II  — w | Q(t)U  <_  |li-w|u  < e. 

Since  c was  arbitrary,  HQ(t)U  ■ YQ(t)CJ  so  IT  e 7 . But  if  V.  c 7 then  7 = 5, 

so  Y e K.  □ 

If  we  allow  F to  be  defined  for  any  vector  y in  Rn,  then  by  a similar 
argument  it  can  be  shown  that  V is  the  set  of  all  vectors,  v,  that  make  F^  a 
homomorphism. 


4 . Examples . 


EXAMPLE  1.  Disney,  Farrell  and  DeMorais  [ 3]  show  that  the  output  of 
an  M/D/1/1  queue  is  a renewal  process.  Tue  results  obtained  thus  far  allow 
for  a quick  verification  of  this  fact. 

The  output  process  from  an  M/D/1/1  queue  is  a two  state  MRP  with  kernel 


Q(t)  - 


( 


^00^ 


Qoi(t) 

Qll(t) 


Qio(t) 
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where  Q„(t)  is  the  probability  that 

(A)  a customer  departs  at  time  zero  leaving  i customers  in  the  queue,  and 

(B)  the  next  departure  is  before  time  t,  and  when  that  customer  leaves 
there  are  j customers  left  in  the  queue. 

If  arrivals  are  Poisson  with  rate  A , and  the  service  times  are  deterministic 
with  rate  d,  then 

0,  if  t < d , 

t-d 

' 

, -As  -Ad  , _ -Ad  -At  _ , 

JAe  e ds  = e -e  , if  t >_  d . 

0 

10  , if  t < d , 
t-d 

J Ae“Xs  (l-e_Xd)ds  = (1  - e~Xd)  (1  - e-X(t-d)),  if  t 21  d . 

0 

0 , if  t < d , 

Ql0<t)  ' « 

e"Xd  if  t >_  d . 

0 , if  t < d , 

1 - e_Xd  , if  t > d. 

Thus, 
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Q(t)  - 


-Ad 


e 


(l-e_X(t-d)) 


-Ad 


(l-e~Xd)(l-e"X(t-d)) 


1 - e 


-Ad 


1 (t) 
(d,») 


where 


1 (t)  = 

(d,<») 


0,  if  t < d, 

1,  if  t > d . 


The  embedded  Markov  chain  is 


so  the  steady  state  vector,  II**  (e  Xd,  1-e 
JIQ(t),  we  get 

IlQ(t)  = ((1  - e_Xt)  e"Xd  , (1  - e"Xt)  (1  - e~Xd 

By  theorem  2.4  we  know  that  Q-r  where  r(t) 


Xd) . Performing  the  multiplication 


))  1 (t)  - (l-e"Xt)  1 (t)n. 

(d,°°)  (d ,°°) 

« (1  - e-Xt)  1 (t)  . 

(d,“) 


EXAMPLE  2.  Burke's  Theorem  [ i ] implies  that  the  output  from  a steady  state 
M/M/1  queue  is  a Poisson  process.  The  output  process  is  a MRP  with  kernel  Q(t) 
where  Q^(t)  is  defined  exactly  as  in  the  first  example,  except  that  in  this 
case  i and  j range  over  all  the  nonnegative  integers.  If  the  service  rate  is 
P,  and  the  arrival  rate  is  A then  Q(t)  has  the  form 


/ 


( 

( 


where 


fj(t) 


I 


C (Xs)^  -Xs  -us  . 

' . , e ye  ds 


J. 


^ pAj  e"(X+li),:  i ^ 


j-k 


(X+u) 


j+1 


k*o  ( j-k) ! (X+y) 


k+1 


tjj  (t)  - J Xe~Xs  f j (t-s)  ds 


(X+y) 


^7  d-.'11)  - X J+1  e'U  J i 


j+1 


i n /I  . xk+l  j-k 

k“0  (X+y)  yJ 


+ 


j J_k 
»J+1  I l 

k-0  p-0 


e~(X+y)t  t j-k- p 
(j-k-p)!  (X+y)k+1  yP 


Because  of  the  special  structure  of  Q(°°),  it  can  be  shown  that  the  steady  state 
-1  2 3 

vector,  II  ■ (1-p)  (l,p,p  ,p  , •••)  where  p » X/y.  Performing  the  multiplication 
IIQ(t)  we  get 
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(IIQ(t)).  - (1-p)"1  (q,(t)  + [ f.  ,(t)  pk+1) 
3 3 k«0  3 K 

= (l-e“Xt)  (1-p-1)  p = (l-e”Xt)H  . 


This  says  that  the  steady  state  output  is  a renewal  process  with  distribution 

..  -At. 

(1-t  ). 


EXAMPLE  3.  Now  consider  the  M/M/l/N  queue.  The  output  from  this  queue 
is  an  N + l state  MRP  with  kernel  Q(t)  = 


q2(t)  ' 

' • qN-l 

f0(t) 

fx(t) 

f2(t)  • 

• • f 

rN-l 

0 

H* 

O 

rf 

V-/ 

fx(t)  • 

* • f 

N-2 

0 

0 

f0(t)  • 

• • f 

N-3 

l Vt> 

d=N-l  3 
00 

l *4<t) 
d*=N-2  3 


f0(t)  l fAt) 
1-1  3 


where  q ^ ( t)  and  f ^ ( t ) are  as  in  the  last  example.  The  steady  state  vector 

_N  1-p  „ 2 3 N-l  . . , N , e~ptpN+1 

II  * nrr  (1,P  P P , • • *,P  ,k) , where  k ■ p + : — . 

l-pN+1  1 “ p 

If  the  steady  state  output  from  the  M/M/l/N  queue  is  a renewal  process 
then  the  renewal  process  would  have  to  have  distribution 

-At  N+l  -Ut 

rN(t)  - n QN(t)U  =1-  - +—  N+2  • 

1-p  1-p 
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Assume  IT  c V.  Then  Vx,  H^Q^(x)  e l^.  Since  V satisfies 

v K EN  Q (x) 

V z l’  =>  ( ^ - E ) J_  QN(t)U,  Yt,  we  must  have  ( — r - T.  ) QR(t)U  - 0,  Vt. 

N 

Q^(t)U  has  the  form  (a,b,b,b,  ,,#,b)  where  a and  b are  positive,  thus 


/ Vx)  n 

( -r^y  - ^ > Vt)u 


( — e,.-  - l)a  + 


1-p1*1  rN(x) 


. „ i ~Xx  K-l  , 


1-0  , 1-€_,'X  K.  -U1CM,  . N*, 

+ 7^T(^W  +e  — )-»  >b  • 


* 

Since  rjo(x)  < 1 - e , each  term  in  the  expression  for 
HNQ  (x) 

( j- r It  ) Q (t)U  is  strictly  positive,  so  n“  t V . Thus  by  theorem  2.8, 

rN  W " 

the  output  of  an  M/M/l/N  queue  is  not  equivalent  to  any  renewal  process.  (See  [3  ].) 


5.  Equivalence  Between  Finite  State  MRP’s.  Let  Q be  a k state  MR P with 

steady  state  vector  H , let  Y be  an  m state  MRP  (m  < k) , and  let  {A, ,A_,  •••  A } 

12  m 

be  a partition  of  the  states  of  Q.  Let  II  be  an  m x k matrix  defined  by 


■ij 


0 , if  j d Af  , 

"j  / 

/ I Ua,  if  j e A , 
aeAj 


and  let  U be  a k * m matrix  defined  by 


■ij 


0 , if  i i A , 


1,  if  i e A, 
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( 


r.Q(t)U  is  an  m * m matrix  whose  (i,j)  element  is  P,-(X^  £ tlx^  £ A^) . 

Say  F is  the  function  that  maps  the  state  space  of  Q to  {A^,A0,  •••.A^}. 

Serfozo  [10]  shows  that  if  {F(Xn),Tn}  is  a MRP,  its  kernel  is  5Q(t)U. 

The  definition  of  equivalence  between  a MRP  and  a renewal  process  has  a 
natural  generalization  to  equivalence  between  two  finite  state  MRP's. 


Definition  2.3.  Let  Q be  a MRP  with  state  space  (1,2,  •••,k)  and  steady 

state  vector  II.  Let  Y be  an  m state  MRP  (m  < k) . Let  F:{l,2,-**,k}  {A-,.***,A  } 

1 m 

be  a partition  of  the  states  of  Q,  and  define  R and  U as  above.  We  say  Q is 

F 

equivalent  to  Y via  the  partition  F,  (Q  - Y)  if  Yn,  t^,*  • *,tn  , 


nQ(t1)  Q(t2)  • • • Q(tn)U  - Y(tx)  Y(t2)  • • • Y(tn) . 

Before  attempting  to  produce  conditions  for  this  type  of  equivalence  we 

should  know  what  this  equivalence  means.  In  the  case  where  Y has  one  state 

(a  renewal  process)  we  showed  that  equivalence  is  the  same  as  weak  lumpability. 

If  Q is  {X  ,T  ) and  Y is  (Z  ,S  ) then  definition  2.3  says  that  for  each 
n n n n 

i,j  e (1,2,  •••,m),  and  yn,  t^  • • ■,  tn. 


(1) 


PIi(Xn  e C1 1 X0  ‘ Ai>  ■ 


For  weak  lumpability  between  Q and  Y we  would  need  that  for  each 
i,j  e {1,2,«  • *,m}  and  Vn,  t^,t2,  • • •,  tn,  i^,i2  *n  p e {1.2,  •••,  m}, 

(2) 


Pyr  {X  £ A . , X . £ A , , * 1 * . X.  e A , ,T  ^ t »•••  »T..  < 

n n J n-1  ij,.!  1 i!»  n - n 1 - 


h'h  E V 


P(Zn*^'  Zn-l'Vl'  Sn-V  ’ * * ’S1  1 h! 1 Z0  * i}  ' 


It  seems  inconceivable  that  every  Q and  Y that  satisfy  (1)  would  also 
satisfy  (2)  but  all  attempts  to  find  a counter  example  have  failed  so  far. 
We  can  show  that  weak  lumpability  implies  equivalence,  though. 


A 
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Since  the  definition  of  equivalence  between  finite  state  MRP's  is 
analogous  to  the  definition  of  equivalence  between  a MRP  and  a renewal  process, 
one  might  assume  that  the  conditions  for  equivalence  would  be  similar.  First 
of  all,  by  theorem  2.13  any  sufficient  condition  for  weak  lumpability  will 
also  be  a sufficient  condition  for  equivalence,  thus  we  have 

Theorem  2.14.  If  Vt,  HQ(t)  = Y(t)n  , then  Q ? Y. 

Theorem  2.15.  If  Vt,  Q(t)U  = UY(t)  , then  Q ? Y. 

Theorems  2.14  and  2.15  can  be  proved  the  same  way  theorems  2.4  and  2.5  were 
proved.  They  can  also  be  found  in  Serfozo  [11]. 

Assume  that  for  each  t,  Y(t)  is  an  invertable  m x m matrix  (m  < ®) , and 

Q(t)  is  a k x k matrix  (m  < k < °°) . Q(t)  need  not  be  invertable.  Let  M be 

the  set  of  all  m x k matrices,  M,  with  M^  >_  0 and  MU  * al  where  n is  a scalar. 
In  other  words  M must  have  the  form 


and  each  row  sum  must  be  the  same. 

If  Y(t)  ^ nQ(t)  d M then  it  must  be  true  that  Y(t)  ^IlQ(t)U  d I which 
says  that  nQ(t)U  + Y(t).  Thus  a simple  necessary  condition  for  equivalence 
is  that  Y(t)-1nQ(t)  E M. 

Let  5 ■ {M  £ M:  MQ(t)U  “ (MU)  Y(t),  Vt}  and  let  V be  the  largest  subset 
of  S that  is  invarient  under  multiplication  by  Y(t)-1  on  the  left  and  Q(t) 
on  the  right  (i.e.  M E V =£>Vt,  Y(t)_1MQ(t)  c V).  Let  K « {M  e V : MU  - I) . 
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These  definitions  are  analagous  to  the  definitions  in  Section  2 and  not  too 
surprisingly  we  have 

Theorem  2.16 . ■ Q - Y <*t~ ~£> H e K . 

Proof.  ( ~ 1>)  Assume  II  i K.  Then  St,  , t~ t such  that 

Y(tn)  ^Y(tn_^)  • • • Y(t^)  1 nQ(t^)Q(t2>  •••  Q(tR)  i M.  But  that  says  3t  so 

that  Y(tn)'1---Y(t1)"1nQ(t1)---Q(tn)  Q(t)U  i YCtj"1  •••  YC^)-1 JJQ(t  ) • • • Q(t  ) 

But  if  Q ~ Y then  Y(tn)  1 •••  Y(t^)  ^nQCt^  • • *Q(tn)U  » I so 

HOCt,)- • -Q(t  ) Q(t)U  # Y(t. ) ***Y(t  )Y(t)  which  is  a contradiction, 
n 1 n 

(•*£--  ) If  u e K then  Vn,  t,  ,t0,«»*,t  Y(t  ) ^•••Y(t.)  1IIQ(t1)  * • -Q(t  )U  ■ I,  so 

- iznn  n - 

nQ(t1)-.-Q(tn)U  - Y(t1)***Y(tn).  □ 

In  general  Y(t)  will  not  be  inver table  for  all  t.  This  poses  a serious 
problem.  In  the  renewal  case  this  problem  did  not  exist  since  r(t)  was  a 
scalar,  not  a matrix.  For  equivalence  we  need  IIQ(t)  * Y (t) II * where  in  some 
sense  II * acts  just  like  n.  Unfortunately,  if  Y(t)  is  singular  we  cannot  solve 
for  n'  uniquely,  and  we  therefore  have  trouble  defining  the  sets  U and  K. 

Another  result  that  carries  over  from  the  renewal  case  rather  easily  is  that 
equivalence  is  identical  to  a certain  ring  homomorphism. 

Let  Q.  be  the  ring  generated  by  {Q(t)J,  t e [0,®],and  let  V be  the  ring 
generated  by  (Y(t)},  t t [0,®].  Define  *n  F:  0 / to  be  4>n  F(A)  - {lAU. 

F 

Theorem  2.17 . Q . Y <—  > 4>^  F is  a homomorphism. 

The  proof  here  is  identical  to  the  proof  in  the  renewal  case. 


One  result  that  has  no  counterpart  in  the  renewal  case  is 
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Theorem  2.18.  Let  Q,  Y and  D be  k,  m and  l state  simple  MRP's  (k  > m > Z) . 

F G G«F 

If  Q - Y and  Y - D then  Q - D. 


To  prove  this  theorem  we  need  the  following  lemma. 

Lemma  2.19.  If  Q ~ Y and  II  is  the  stead}’  state  vector  for  Q then  the 

steadv  state  vector  for  Y is  y where  y « I . H (i.e.  y ■ HU). 

1 pcF-i (i)  P 

Proof.  Since  the  steady  state  vector  is  unique  in  the  class  of  simple 
MRP's,  it  suffices  to  show  that  Y satisfies  Y Y(°°)  * Y. 

Since 


-ij 


0,  if  j i F"A(i), 
n 


I n 

PCF-I(i)  p 


3 , if  j e F-1(i), 


we  have 


yY(®)  * y^Q(°°)U  ” HQ(®)U  * ITU  * y*  D 


Proof  of  Theorem.  Let  j]  be  the  steady  state  vector  for  Q,  Y the  steady 
state  vector  for  Y and  let  F.p  and  Yq  be  the  matrices  induced  by  the  partitions 
F and  G.  We  first  show  that  Yq5f  * ?g°F  w*lere  5g0p  is  t*le  £ x k matrix  induced 
on  II  by  the  partition  G«F.  First  of  all,  if  G«F(j)  i i then  (YgHp)^  ■ 0. 

If  G°F(j)  « i then  since  each  column  of  Yq  and  lip  has  only  one  nonzero  element, 
there  is  some  b such  that  (Ygllp)^  * ^YG3ib^*F3bj’  Froffi  lefflIna» 


(YG}ib 


l 1 

aeF_1(b) 


L i 

atF"1(b) 


I ' 

pcG-l(i) 


l 


l. 


peG“l(i)  qeF-1(p) 


I . 11 

qc[GoF]_1(i) 
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Thus, 


(v.y  . • = 

'G'F  ij 


I i 

aeF~1(b) 


l 


l . nq 

qt [G°F]“l(i) 


l na 

acF-1 (b) 


l T- 

qe[GoF]_1(i)  q 


Thus , 


(v  Tt  \ 

~G~F'  ij 


0,  if  GoF(j)  * i, 

, if  G°F(j)  = i , 


l n 

i q 

qs [G°F ] 1(i) 


TG5f  = 5GeF  as  desired. 

If  we  let  Up  be  the  k x m summing  matrix  associated  with  F,  and  be 
the  m x £ summing  matrix  associated  with  G,  it  is  easy  to  show  that 
U_U-  = where  U_  _ is  the  k x £ summing  matrix  associated  with  G°F. 

•r-u  *’b°r 

Since  Q ■ Y,  Vn,  tp,  t2*--tn,  3pQ(tj.)  * * = Y(t, ) • • *Y(tn) . 


n VF 


Also,  since  Y - D,  Y_Y(t,)***Y(t  )U_  * D(T1)D(t,)**»D(t  ).  Thus 
■'’Ox  no  x z n 

YgTipQCtj^)  • • •Q(tn)UFyG  - D(tp)  • • -D(tn)  . But  this  says 
5GoFQ(tl)"’Q(tn)UG.F  " D(tl)“"D(tn)»S0  Q G‘F  D>  ° 

There  are  several  open  questions  on  the  topic  of  equivalence  between 
finite  state  MRP’s.  Among  them  are  finding  an  example  of  two  finite  state 
MRP’s,  Q and  Y,  that  are  equivalent  but  not  weakly  lumpable;  and  finding 
conditions  for  equivalence  when  the  matrices  Y(t)  are  allowed  to  be  singular. 
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If  Q - Y and  Y - D,  then  we  have  shown  that  0G-FD.  Now  suppose  Q ~ Y and 
Q D.  Under  what  conditions  is  there  a partition,  H,  such  that  Y ? D? 

In  other  words,  when  is  there  an  H that  makes  the  following  diagram  commute? 


MRP  and  a renewal  process.  Say  Q is  an  n state  MRP  with  steady  state  vector  ", 


r is  a renewal  process  and  Q ~ r.  Let  Q'  be  another  n state  MRP  that  has  the 
property  that  t £ T implies  that  Q(t)  *=  Q'(t).  If  ^ T i*l,2,"'k  then 
EQ(t^)  • • -Q(tk)U  = rCtj^)  • • *r(tk)  . But  since  Q(t)  ■ Q'(t)  for  t < I we  also 
have  IIQ' (t^) • • ‘Q’ (tfc)U  « r(t^) • • -r (t^) . Although  it  is  possible  that  Q'  * r 
(in  fact  I!  might  not  even  be  the  steady  state  vector  for  Q')  we  do  have  a sort 
of  equivalence  between  Q'  and  r on  [0,T]. 

The  motivation  behind  this  section  is  that  in  general  a MRP,  Q,  is  not 
equivalent  to  any  renewal  process  and  therefore  computing  probabilities  of 
successive  interdeparture  times  involves  matrix  multiplication.  If  there  is 
some  BC  R such  that  if  t^tg,*'  ,tk  e B then  P^  <_  t1,**-fTk  ^ t^)  ■ 

P(T^  t^)  P(T^  ^ t2)  • • • P(T^  ^ t^) , computing  probabilities  would  only  involve 
scalar  multiplication  (so  long  as  all  the  times  are  chosen  from  B) . Also,  it 
seems  interesting  to  consider  the  concept  of  a MRP  acting  like  different 
renewal  processes  on  different  subsets  of  [0,®]. 


Definition  2 ■ L . Let  Q be  an  n state  MRP  and  let  r be  a renewal  process. 


We  say  Q - r on  A if  3y  such  that  y _>  0 and  Vk,  and  Vt^,**%  t^  c A, 
yQCtj) ' * -Q(tk)U  = rftj) • • -r(tk) . 

Notice  that  if  Q * r op  A then  Q - r'  on  A for  any  r'  that  satisfies 
r'(t)  = r(t),Vt  e A.  For  an  arbitrary  MRP,  Q,  one  can  always  construct  a 
set  BCR  such  that  Q - r on  B for  some  r (although  sometimes  B will  consist 
of  only  one  point).  Choose  some  t e [0,°°J.  By  the  Perron-Frobenius  theorem, 
there  is  a largest  positive  eigenvalue,  a(t)  _<  1,  of  0(t)  and  an  associated 
eigenvector  v(t)  that  satisfies  v(t)U  = 1.  Let  B ■=  {s:  v(t)  0(s)  *=  a(s)  v(t)}  . 
Clearly  Q - a on  B. 

Definition  2.5.  If  {A  },  p e J,  is  a cover  of  [0,°°]  (i.e.  UA0  = [0,-]), 

B g B 

and  for  each  6 t J there  is  a renewal  process  rg  such  that  Q * r^  on  A^ 
then  call  {A  } a renewal-cover  of  Q. 

P 

Such  a cover  exists  for  any  Q since  if  necessary  there  can  be  a different 
renewal  process  for  each  t e [0,°°].  The  interesting  question  is  whether  or 
not  one  can  find  a finite  or  countable  renewal-cover.  If  Q - r then  there  is 
a renewal-cover  with  one  element  (i.e.  Q - r on  [0,°°]). 

Theorem  2.20.  Say  Q is  a finite  state  MRP  such  that  Q^(t)  is  a continuous 
function  of  t for  each  i and  j.  Let  a(t)  be  the  largest  positive  eigenvalue  of 
Q(t)  and  let  v(t)  be  its  associated  nonnegative,  normalized  eigenvector.  If 
the  set  (v(t)},  t e [0,°°],  is  finite  or  countable  then  so  is  the  smallest 
renewal-cover,  and  there  exists  an  open  interval  AC  [0,°°]  such  that  Q - a on  A. 

Proof.  Let  {v1,V2>,“}  be  a list  of  the  elements  of  (v(t)}  and  let 

A * (s:  v Q(s)  « a(s)v  }.  (A  } . 0 is  the  desired  renewal-cover.  Say 

n n n n nEl,z,»»' 


• c A and 
n 


- t.  Since  Q(t)  is  continuous  lixn  vnQ(t^)  * v^O 

i -►  oc 

lim  ci ( t . ) v * a(t)v  . Thus  v O(t)  ■ a(t)v  so  A is 
n n n n 


closed.  By  the  Baire  Category  Theorem  one  of  the  Ar  must  contain  an  open 
interval.  o 
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